This invention relates to an optical system which produces a registered pair of images of a scene, such as a robot workspace, one a conventional gray scale, luminance based image and the other a range image.
Two of the major tasks for a machine vision system are object recognition and object pose definition. Pose is defined as the position and orientation of an object in space. Past work in machine vision has stressed computer processing of two-dimensonal luminance images similar to TV images. Some objects are easily recognized in a gray scale image due to fairly reasonable optical contrast between that object and other objects or the background on a scene. Other objects whose contrast is not as pronounced are not as easily discriminated. Recent research activity has focused on generating range images of a scene or group of objects in the field of view. A range image is a two-dimensional array of numbers each of which represents the distance from a given reference point or plane within the range camera to a point physically located on the surface of the object being imaged. There is currently a lot of interest in developing approaches to processing range images in order to recognize an object in the scene, as well as to determine its position and orientation in space.
Each type of image has its own appropriate type of processing. If an object is similar in color and brightness to other objects in its proximity, it may be difficult to isolate it based solely on a two-dimensional luminance view, whereas it may be quite easy to isolate and recognize it based on its size or volume. Similarly, an object may not have a strong three-dimensional nature and so not be easily visible in a range image, but it may strongly differ in color or contrast with its surroundings. Thus, it may be advantageous to be able to simultaneously examine and process the range image and the luminance image to recognize and locate objects. It is probable that by using appropriate processing in each image, the total task can be accomplished faster, cheaper and more easily than if one were to rely solely on either image type.
The problem is how to effectively generate the registered range/luminance images. Optical means using beam splitters and other elements have been used to allow two distinct sensors to be aimed along a common axis. Unfortunately, extensive mechanical setup time and effort are required, and complicated calibration techniques must be used and there is the ever present problem of drift between the two sensors. Maintaining accurate long term alignment is difficult. What is needed is a range imaging technique that allows measurement of reflectance, or surface brightness, using a single common sensing element. Alternatively, one may wish to use a luminance oriented technique that would allow for measurement of range using a single common sensing element.
Since range camera development is a relatively new field there have not been many attempts at registered range/luminance cameras. Most approaches to range measurement employ a structured light technique or a time-of-flight point range sensor which is scanned over the scene. In the structured light case a large beam splitter is utilized to combine the range detection system optically with an ordinary TV-type luminance camera. A similar technique is to have the two cameras aimed at a common point which is fairly distant; the angle between the two views is small and the images nearly coaxial. The second class of device, point rangers using time-of-flight, has been combined in an intrinsic way with reflectance measurement as shown in the published paper by D. Nitzan et al, Proceedings of the IEEE, February 1977. The strength of the return is proportional to the surface reflectance. The reflectance map can indirectly yield the equivalent of a luminance image under given assumptions about lighting, etc.
The three-dimensional range camera in this application is described in the inventor's copending application with that title, Ser. No. 717,367 filed Mar. 28, 1985, the disclosure of which is incorporated herein by reference.